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Introduction 

Jacobians of curves are the best understood abelian varieties. There are many geometric ways of 
constructing curves in jacobians whereas it is difficult to construct interesting curves in most other 
abehan varieties. In this paper and its sequels we introduce methods for determining whether a 
given curve in a jacobian deforms with it when the jacobian deforms to a non-jacobian. We apply 
these methods to a particular class of curves in a jacobian (see below) . One of our motivations is the 
Hodge-theoretic question of which multiples of the minimal cohomology class on a given principally 
polarized abelian variety can be represented by an algebraic curve (see |8j for other motivations). 
In the known examples, the construction of such curves often also leads to parametrizations of 
the theta divisor. Let us begin by summarizing some of what is known about this question. 

For a principally polarized abelian variety (ppav) {A, B) of dimension g over C, let [O] G 
H'^{A,'Z) be the cohomology class of O. The class ^^z^ is called the minimal cohomology class 
for curves in {A, O). We will assume g > 4, since otherwise all multiples of the minimal class are 
represented by algebraic curves. 

If [A, 6) = {JC := Pic^C, G) is the jacobian of a smooth, complete and irreducible curve C of 
genus g, the choice of any invertible sheaf C of degree 1 on C gives an embedding of C in JC via 

C ^ JC 

p ^ Ocip)®C-\ 

Such a map is called an Abel map and the image of C by it an Abel curve. The cohomology class 
of an Abel curve is the minimal class ^Ziyr- By a theorem of Matsusaka J2|, the minimal class 
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is represented by an algebraic curve C in (A, O) if and only if [A, 0) is the polarized jacobian 

( JC, 0) of C. The sum oi g — 1 Abel curves in JC is a theta divisor. 

In Prym varieties even multiples of the minimal class are represented by algebraic curves: 
For an etale double cover vr : C ^ C of smooth curves {C of genus g + 1), the involution 

a : C C oi the cover acts on the jacobian JC and the Prym variety P of vr is defined as 

P := im{a - 1) C JC. 

The intersection of a symmetric theta divisor of JC with P is 22 where the divisor S defines 
a principal polarization on P. Via a — 1, the image of an Abel embedding of C in P gives an 
embedding of C in P and its image is called a Prym-embedded curve. The class of a Prym- 
embedded curve is 2 ■ An interesting question is when, in a Prym variety, are any odd 
multiples of the minimal class represented by algebraic curves. 
A further generalization is the notion of Prym-Tjurin variety: 

Suppose there is a generically injective map from a smooth complete curve X into a ppav P 
with theta divisor S such that the class of the image of X is mj^^Yy,. This yields a map JX P 
with transpose P JX. We say (P, S) is a Prym-Tjurin variety for X if the map P JX is 
injective. This is equivalent to the existence of an endomorphism (p : JX —>■ JX such that 

P = P(X,0) := 2m(0- 1) C JX and (0 - 1)(0 - 1 + m) = 

where the numbers denote the endomorphisms of JX given by multiplication by those numbers. 

Welters showed that every principally polarized abelian variety is a Prym-Tjurin variety. Birken- 
hacke and Lange showed that every principally polarized abelian variety is a Prym-Tjurin variety 
for an integer m < 3^{g — 1)! (see |3] page 374 Corollary 2.4)^. 

The set of integers n such that n-^^jy,- is represented by an algebraic curve (union {0}) is a 
semi-subgroup of Z. There is therefore a unique minimal set of positive integers {niQ < . . . < } 
such that ^^rjy]- is represented by an algebraic curve if and only if either n = mf for some i or 
n > is a multiple of dA where dA is the gcd of the mf. These various integers can be used 
to define stratifications of the moduli space Ag of ppav. The stratification associated to uiq is 

^Their proof uses 3-theta divisors. Using the fact that a general 2-theta divisor is smooth, the exact same 
proof would give m < 2^{g — 1)!. For abelian varieties with a smooth theta divisor, the same proof would give 
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related to a stratification of Ag by other geometric invariants as discussed by Beauville [2] and 
Debarre jSj- Debarre proved in jB] that is at least ~ i (^) ®) general. 

In this paper and the next we use first order obstructions to deformations to identify certain 
families of curves which could potentially deform to non-jacobians. Our method can be applied 
to subvarieties X of JC contained in many translates of the theta divisor. For an integer e > 2 
let C^^^ be the e-th symmetric power of C. Choose a E Pic^~^C and define p as the map 

p: C^9-i) — , JC 

D ^ Oc{D)®a~^ 

whose image is a theta divisor, say 0^. The idea is to use Green's sequence (see Section 

Tc„-i) p*Tjc Tz.^A^a) — > 

where Zg_i is the locus where p fails to be an embedding, the letter T denotes tangent sheaves and 
the letter X ideal sheaves. Given an infinitesimal deformation r] G H^{Tjc), the curve X deforms 
with JC in the direction of rj if and only if the image of rj by the first order obstruction map 

u : H\Tjc) H\Nx/jc) 

where Nx/jc is the normal sheaf to X in JC, is zero (see Section |21). As we shall see in Section 
El the map 

factors through u. It follows that if, for some a, the image of r] in H^(Xzg_j^{Qa)\x) does not 
vanish, then ulrj) does not vanish either. For the examples that we have chosen (see below) we 
prove the stronger statement that the image of t] by the map 

H\Tjc) ^ H\lz^_,nx{ea)) 
is not zero. This map factors into the composition (see Section 

H\Tjc) ^ H\Oz,_,nxm ^ H\lz,_,nxm). 

We analyze these two maps separately in Sections El and El Section El (the Appendix) contains 
some useful technical results. 

The curves that we have chosen for the illustration of the above method are the natural gener- 
alizations of smooth Prym-embedded curves in tetragonal jacobians. More precisely, let C be a 
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curve of genus g with a g\ (a pencil of degree d). We define curves Xe{g]j) whose reduced support 
is 

Xe{gl)red ■= i^e : 3D G C^"^"^) such that D, + Deg\}(l C(^) 

(for the precise scheme-theoretical definition see 12.11 when e = 2 and jH] for e > 2). If (i > e + 1, 
then Xe{g^) can be non-trivially mapped to JC by subtracting a fixed divisor of degree e on C. 
Given a one-parameter infinitesimal deformation of the jacobian of C out of the jacobian locus J^g 
we ask when the curve Xe{g^) deforms with it. In this paper we prove the following 

Theorem 0.1. Suppose C non-hyperelliptic and d > 4. Suppose that the curve X := X2{g]} 
deforms in a direction rj out of Jg. Then 

(1) either d = 4 

(2) or d = 5, h^{g\) = 3, C has genus 4, L is base-point-free, the curve C has only one g\ 
with a triple ramification point t such that bt E L (Z \K — t\ and X2{g\) meets X only at 
2t with intersection multiplicity 4. 

For d = ?) the image of X2{g\) in JC is an Abel curve, hence cannot deform out of Jg ^]. For 
(i = 4, the curve X2{gX) is a Prym-embedded curve hence deforms out of Jg into the locus 
of Prym varieties. For = 5, h^{g]) = 3 and (7 = 4 (with only one gl) or = 5 it is likely that 
X2{gl) deforms out of Jg. An interesting question is to describe, as geometrically as possible, the 
deformations of {JC, 0) with which X2{gl) deforms. 

For e > 2, the analogous result would be the following. The curve Xe{gl) deforms out of Jg 
only if 

• either e = h^{g}) and d = 2e 

• or e = /i°(fi'^) - 1 and d = 2e + 1. 

We prove this in |H] for e < g — 3 under certain hypotheses of genericity. This shortens the list of 
the families of curves whose deformations we need to consider. For more details see |H|. 

So we have some families of curves which could possibly deform to non-jacobians. We need a 
different approach to prove that higher order obstructions to deformations vanish: this will be 
presented in detail in the forth-coming paper jH] and the idea behind it is the following. For each 
Qa containing X, one has the map of cohomology groups of normal sheaves 

H\Nx/jc) H\Nejjc\x) = H\Ox{Qa)) 
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whose kernel contains all the obstructions to the deformations of X since we will only consider 
algebraizable deformations of JC for which the obstructions to deforming vanish. If one can 
prove that the intersection of these kernels is the image of the first order algebraizable deformations 
of JC, i.e., the image of S^H^{Oc) C H^{Tjc), it will follow that the only obstructions to 
deforming X with JC are the first order obstructions. 

Finally, we would like to mention that from curves one can obtain higher- dimensional subvari- 
eties of an abelian variety. For a discussion of this we refer the reader to [TH] . 

1. Notation and Conventions 
We will denote linear equivalence of divisors by ~. 

For any divisor or coherent sheaf D on a scheme X, denote by h'^{D) the dimension of the 
cohomology group H^{D) = H^{X, D). For any subscheme Y of X, we will denote Ty/x the ideal 
sheaf of y in X and Ny/x the normal sheaf of y in X. When there is no ambiguity we drop the 
subscript x from 2y/x or Ny/x- The tangent sheaf of X will be denoted by Tx := Ttom{flx, Ox) 
and the dualizing sheaf of X by ujx- 

We let C be a smooth non-hyperelliptic curve of genus g over the field C of complex numbers. 
For any positive integer n, denote by C" the n-th. Cartesian power of C and by the n-th 
symmetric power of C. We denote 7r„ : C" C-"^ the natural quotient map. Note that 
parametrizes the effective divisors of degree n on C. 

We denote by K an arbitrary canonical divisor on C. Since C is not hyperelliptic, its canonical 
map C \K\* is an embedding and throughout this paper we identify C with its canonical image. 
For a divisor D on C, we denote by (D) its span in \K\*. 

Since we will mostly work with the Picard group Pic^~^C of invertible sheaves of degree g — 1 
on C, we put A := Pic^~^C. Let G denote the natural theta divisor of A, i.e., 

G := {£ G A : h^{C) > 0} . 

The multiplicity of £ G G in G is h'^{C) Chapter VI p. 226). So the singular locus of G is 

Sing{e) ■.= {CeA: h\C) > 2} . 

There is a map 

Stng^iO) \hiC)\ 

C ^ Q{C):=Une\c\{D) 
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where Sing2{Q) is the locus of points of order 2 on O and |/2(C)| is the linear system of quadrics 
containing the canonical curve C. This map is equal to the map sending C to the (quadric) tangent 
cone to 9 at £ and its image Q generates |/2(C')| (see jTj and [H]). Any Q{C) G Q has rank < 4. 
The singular locus of Q{C) cuts C in the sum of the base divisors of \C\ and \ujc ® C~^\. The 
rulings of Q cut the divisors of the moving parts of \C\ and \u!c ® on C (see [Ij). 

For any divisor or invertible sheaf a of degree and any subscheme Y of A, we let Ya denote 
the translate of Y by a. By a we mean a (not necessarily complete) linear system of degree 
d and dimension r. We call WJj^ the subvariety of Pid^C parametrizing invertible sheaves C with 
h^{C) > r. 

For any effective divisor E of degree e on C and any positive integer n > e, let C C*^"^ be 

the image of C*^"""^^ in C'-"^ by the morphism D ^ D + E. Write Ct '■= C*f^\ and for any divisor 
E = X]I=i ^i^j C*, let Ce denote the divisor X]i=i "-jC't. on C*^^\ For a linear system L on C, 
we denote by any divisor Ce C C^^^ with E E L. We let S denote the divisor class on C*^^^ 
such that TTgi^ ~ A where A is the diagonal of C^. 

By infinitesimal deformation we always mean flat first order infinitesimal deformation. 

2. The curve X and the first order obstruction map for its deformations 

2.1. Let L be a pencil of degree d > A on C. Let M be the moving part of L and let B be its 
base divisor. Define the curve X := X2{L) as a divisor on C*^^-* in the following way 

X = X2{L) := Xm + CbC 

where Xm '■= X2{M) is the reduced curve 

Xm = X2(M) := {D2 : 3D E C^''-^) g^ch that D2 + D E M}. 

Lemma 2.1. VFe have 

X ~ — 5 

and X has arithmetic genus 

{d - 2){2g + d - 3) 
9x- . 

Proof. Pull back to C^, restrict to the fibers of the two projections and use the See-Saw Theorem. 
For the arithmetic genus use the exact sequence 
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and the results of Appendix 16.11 □ 

2.2. Choose (7 — 3 general points pi,...,Pg_3 in C and embed C*^^-* in C^^^^^ and A by the 
respective morphisms 

C(2) ^ ^(3-1) ^ A 

Identify X and C^^^ with their images by these maps. 
Recall the usual exact sequence 

'Ix/aI1\/a — ^ ^aU — ^ — ^ 0- 

The curve X is a local complete intersection scheme because it is a divisor in C^'^\ Using this, 
local calculations show that the above sequence can be completed to the exact sequence 

' '^Xr,d/X ■ '^x/a/'^x/a — ' '^x/A/ix/A — ' ^\\x — ^ — ' 

where X^ed is the underlying reduced scheme of X. This sequence can then be split into the 
following two short exact sequences. 

(2.1) — ^ 1xr,a/x ■ Tx/A/ijc/A — ^ '^x/a/1\/a — ^ ^T^/A A^/a U,,d — ' 

^ lx/A/l\/A\Xr,a ^ ^Wx ^ ' 

from which we obtain the maps of exterior groups 

Ext\lx/A/1%A\x,.^..Ox) — H\Nx,a) 

H\TA\x)-^Ext\lx/A/ll/A\x^^,.Ox). 
The composition of the above two maps with restriction 

H\Ta) H\Ta\x) 

is the obstruction map 

u:H\Ta)-^H\Nx/a). 

Given an infinitesimal deformation rj G H^{Ta), the curve X deforms with A in the direction of rj 
if and only if //(r/) = (see e.g. ^T] Chapter 1 and [16] for these deformation theory results). 
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2.3. Using the fact that X is a divisor in C^^\ a local calculation shows that we have the usual 
exact sequence 



— 2^c(2)/a/2^c'(2)/aU — '^x/a/I'x/a — ^x/c(2)/^x/c(2) — ^ ^ 
whose dual is the exact sequence 

— )• Nx/cC2) — Nx/A — Nq(2)/j^\x — 0. 
From this we obtain the map 

H\Nx/A)^H\Nci2yM 
whose composition with u we call 1/2' 

V2: H\Ta) ^ H\Nci2yA\x). 

So, if v{r]) — 0, then, a fortiori, ^'2(^7) — 0. 

2.4. The choice of the polarization provides an isomorphism H^[Ta) = -ff^(Oc)®^ by which 
the algebraic (i.e. globally unobstructed) infinitesimal deformations with which © deforms are 
identified with the elements of the subspace S^H^{Oc) C H^{Oc)^^ ^ H\Ta). Via this identifi- 
cation, the space of infinitesimal deformations of {A, ©) as a jacobian is naturally identified with 
H^{Tc) C S'^H^{Oc)- The Serre dual of this last map is multiphcation of sections 

S''H\K) H\2K) 

whose kernel is the space hiC) of quadrics containing the canonical image of C. Therefore, to 
say that we consider an infinitesimal deformation of (A, 0) out of the jacobian locus, means that 

we consider 77 G S'^H^iOc) \ H^{Tc) which is equivalent to say that wc consider rj G S^H^{Oc) 
such that there is a quadric Q e hiC) with {Q,r]) 7^ 0. Here we denote by 

(, ) : S^H\K) ® S^H\Oc) S^H\K) 

the pairing obtained from Serre Duality. 

We fix such an infinitesimal deformation 7] and prove that if i'2{fi) = 0; then d = 4 or 0? = 5 and 
/i°(L) = 3. For this we use translates of containing C^^^ and, a fortiori, X. 
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3. The translates of 6 containing C^^^ d X and the first order obstruction map 

Lemma 3.1. The surface C^"^^ is contained in a translate Ga of O if and only if there exists 
^ gi G C*^^^^^ such that a = ^Pi — ^qi- 

Proof. For any points gi, . . . , g^-a of C, the image of C^^^ in A is contained in the divisor 9^p._^ q^. 
Conversely, if C*-^-* is contained in a translate Ba of B, then we have h^{D2 + '^Pi — cl) > 0, for 
all D2 E C^^\ Equivalents, for all D2 E C'^^\ we have h\K + a - ^p, - D2) > 0, i.e., 
h^{K + a — ^pi) > 3 and —a + J2Pi is effective. □ 

3.1. Choose Oc{a) E Pic^C such that C^'^^ C B_a (i.e., —a = J2Pi ~J2Qi above). Then 
cP C B. Let p : C(f-i) ^ B be the natural morph ism. Then (see jZj (1.20) p. 89) we have the 
exact sequence 

(3.1) T^(,-i) p*Ta Xz,_,(B) 

where the leftmost map is the differential of p and Zg^i is the subscheme of C^^^^^ where p fails 
to be an isomorphism. For the convenience of the reader we mention that the scheme Zg^i is a 
determinantal scheme of codimension 2. If (7 > 5 or if (7 = 4 and C has two distinct gl^s, the 
scheme Zg_i is reduced and is the scheme-theoretical inverse image of the singular locus of B. 

Combining sequence ()3.1|) with the tangent bundles sequences for c!^\ we obtain the commu- 
tative diagram with exact rows and columns 





1 i 

T„(2) = T (2) 

i i 

i i II 

c(^Vc(«-i) ^ ^c'/^/A ^2,-i(0)lcP) ^ 

i i 
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where the leftmost horizontal maps are injective if and only if h^iJ^Qi) — 1- The restriction of 
this to Xa gives the commutative diagram with exact rows and columns 

i i 

i i II 

i i 



whose cohomology gives the commutative diagram with exact rows and columns 

i i 

i i II 

i i 


Therefore we have the commutative diagram 

U \ i II 

Translation by a induces the identity on H^{Ta) and isomorphisms 

H\Ta\xJ = H\Ta\x) //^(7V^(2)/^|xJ = H\Nci.yA\x) 
so that the kernel of 

U2 : S^H\Oc) H\Nc^2),a\x) 
is equal to the kernel of the map 

obtained from 1/2 by translation. Therefore the previous diagram proves the following theorem. 
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Theorem 3.2. The kernel of the map 

V2 : S^H\Oc) H\Nci2,,A\x) 
is contained in the kernel of the map obtained from the above 

S'H\Oc) ^ H\lz,_,mxJ 
for all a such that 6_o contains C^'^K 

3.2. We shall prove that for any r] e S^H^{Oc) \ H^{Tc), there exists a such that 0_a contains 
C^^^ and the image of rj by the map 

is nonzero unless either d — A or d — 5, h^{L) — 3 and C has genus 5 or genus 4 and only one g^. 

3.3. The latter map is the composition of 

with restriction 
Prom the natural map 

Xz,_,(e)|x. — Xz,_,nx.(e) 

we obtain the map 

Therefore we look at the kernel of the composition 

S'H\Oc) H\lz,_,{e)) H\lz,_,mxJ 

(3.2) 

^ H\lz^_,nxSQ)). 

Prom the usual exact sequence 

Xz,_,(e) Oc.-^{e) Oz,_,{e) o, 

we obtain the embedding 

H%OzU^))-^H\lz,_,{e)). 



12 E. IZADI 

By 17] p. 95, the image of S^H\Oc) in H\Iz^_^{Q)) is contained in i7°(C»z,_i (O)). Now, using 
the commutative diagram with exact rows 

IzU^) Oc.~m OzU^) 

■J- ■I' -l- 

Composition 13.21 is equal to the composition 

S'H\Oc) H\Oz,_AQ)) H%Oz,_,nxSQ)) 

By |Z] p. 95 again, the first map is the following 



S^H\Oc) ^ H\Oz,_AQ)) 



where {zi} is a system of coordinates on A and cr is a theta function with divisor of zeros equal 
to 9. So we have 



coboundary ttI/t /r\\\ 
> rl [IZg-inXaV^j) 



^9- 
? 



We will investigate the kernel of the composition of the first two maps and that of the coboundary 
map separately. The kernel of the composition of the first two maps is contained in (with equality 
if and only if Zg_i fl Xa is reduced) the annihilator of the quadrics of rank < 4 which are the 
tangent cones to at the points of p(Zg_i) fl Xa = SingiQ) fl Xa. 

4. The kernel of the map S^H\Oc) i/°(Oz,^inx„(0)) 

4.1. Let Z(X) C C^^^^^ X X be the closure of the subvariety parametrizing pairs g^, D2) such 
that h^C^ Qi) = 1 (this is the case generically on C^^~^^ because dimiyj_3 <g — 3 — 2 — 1 = g — 6 
by [13 PP- 348-350) and h^{D2 + E^i) = 2. Let Z{X) C C^^-^) be the image of Z{X) by the 
first projection. Denote by Z{Xm) and Z{Xm) the corresponding objects for Xm- It follows from 
Corollarv 14.31 below that Z{X) and Z{X) are not empty and Z{Xm) and Z{Xm) are not empty 
when either the degree of M is at least 4 or g > 5. 
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Given an infinitesimal deformation t] G S'^H^{Oc) \ H^{Tc) we sliall prove tliat tfiere is always 
a component of Z{X) such that for {^qi,D2) general in that component the tangent cone to G 
at Oc{D2 + ^Qi) does not vanish on rj. This will follows from Corollarv 14.31 below, given that 
the image Q of Sing2{Q) in |/2(C)| generates |/2(C)|. By our remarks above, it implies a fortiori 
that the image of rj in H^{Ozg_inXa{^)) is nonzero for —a = YliPi~Yli'ii- We will then show that 
generically on any component of Z{X) the coboundary map 

is injective unless either d = 4 or d = 5 and h^{L) = 3. It will follow by Theorem 13.21 that 
^2{v) 7^ 0) hence 1^(77) 7^ and X does not deform with t] unless either d = 4 or ci = 5 and 
h^{L) = 3. We begin by computing the dimensions of Z{X) and Z{X) and showing that Z{X) 
maps onto Sing{Q). 

Lemma 4.1. For any quadric Q containing C , there exists D2 E X such that (-D2) C Q. If either 
g > 5, or the degree of M is at least A, then such a D2 can be chosen to be in Xm C X. 

Proof. By Appendix l6.3l the space /2(C) can be identified with H^{C^'^\ Ck—'^S). So Q corresponds 
to a section sq E H'^{C^'^\Ck — 25). The support of the divisor Eg of zeros of sq is the set of 
divisors D E C*^^-* such that (D) C Q. So our lemma is equivalent to saying that the intersection 
Eq n X is not empty. This follows from the following computation of the intersection number of 
Eq and X where we use d > 4 and (? > 4. 

X-Eq = {Cl-6)- {Ck - 25) = d{2g -2) -2d- {2g - 2) - 2{g - 1) 

= d{2g - 4) - 4^ + 4 > 4(2^ - 4) - 4c/ + 4 = 4^ - 12 > 4. 

The analogous calculation with Xm instead of X proves the second asertion. □ 

Lemma 4.2. For any g]^_^ on C , there exists D2 E X such that h^{gg_i — -D2) > 0. // either 
g >5, or the degree of M is at least 4, then such a D2 can be chosen to be in Xm C X. 

Proof. This follows from the positivity of the intersection number of X and X2{g]i_i): 
X.X2{gU) = {CL-5)-{Cg._^-5)=d{g-l)-d-{g-l)-{g-l) 
= d{g - 2) - 2g + 2 > A{g - 2) - 2g + 2 = 2g - Q > 2. 
Tha analogus calculation with Xm instead of X proves the second assertion. □ 
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Corollary 4.3. The variety Z{X) maps onto Sing{Q) by p. If either g > 5, or the degree of M 
is at least 4, then Z{Xm) also maps onto Sing{Q) by p. 

Proof. It is sufficient to prove that the map Z{X) Sing{<d) is dominant. A general point of 
Sing{Q) is a complete g^^i on C. By the previous lemma, there is a divisor of gg_i which contains 
D2. So there is ^ G C^^s) ^j^h D2 + ^qie gl_^. The pair g^, D2) G C^^s) x x maps to 
gl-i by p. To see that qi, D2) G Z{X) for a general choice of g^^i, it is sufficient to prove that 
gj) = 1 for a general choice of g]j^i- 

If g]j^i is base-point-free, then this is automatic. If g]j^i has base points, then it is sufficient 
to prove that no divisor D2 X is contained in its base divisor. It follows from a theorem of 
Mumford (see |2] p. 193) that a general g]^^^ is base-point-free unless C is either trigonal, bieUiptic 
or a smooth plane quintic. In all these cases, the base divisor can be chosen to be general so that 
it contains no divisor D2 G X. 

The assertion about Z{Xm) is proved similarly, using the corresponding statements for Xm- □ 

Proposition 4.4. The varieties Z{X) and Z{Xm) (when non-empty) are everywhere of dimension 

Proof. By Corollary 14. 3| the variety Z{X) is not empty. To see that the dimensions of Z{X) and 
Z{Xm) are everywhere > (7 — 4, note that h^{D2 + X] — 2 is equivalent to D2 + J2 ^ ^g-i- 
Requiring D2 E X (resp. Xm) imposes at most one condition on the pair (^gj,D2)- Since the 
dimension of Zg^i is g — 3 [7], the proposition follows. 

□ 

4.2. Since quadrics associated to gl^i's generate hiC) (see |7] and [IHl), for any direction rj G 
S'^H^{Oc) \ H^{Tc) there exists an irreducible component Q{ri) of Q such that for Q general in 
2(77), the quadric Q is nonzero on 77 (in fact Q is almost always irreducible but we do not need to 
go into this). Let Z{ri) be an irreducible component of Z{X) which maps onto Q{r]) and let Z{r]) 
be the image of Z{r]) in C'^^^^\ If the degree of M is at least 4 or if (7 > 5, choose Z{r]) and Z^i]) 
to be in Z{Xm) and Z{Xm) respectively. Then, for general in Z{ri), the image of rj in the 
corresponding H^{Ozg^inXa{^)) is nonzero. 
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5. The coboundary map H%Oz,_,nxSQ)) ^ H\Iz^_,nxSQ)) 
Lemma 5.1. Suppose 'Y^qiE Z{X) satisfies hPi^Qi) = 1- If the coboundary map 

is not injective, then 

H\C,K-Y,q,-L)^Q. 

Proof. Using the exact sequence 

^ H\lz,_,^xAm — H\OxAQ)) 
— H\Oz,_,r.xS^)) — H\Xz,_,nxAQ)). 
we need to understand the sections of which vanish on Zg^i\~\Xa. Equivalently, translating 

everything by —a, we need to understand the sections of (9x(0-a) which vanish on {Zg_i)_a HX. 
For this, we use the embedding of X in C'-^^: 

Since Oci2){Q.a-X) = 0^(2) (C^x-e^.-l) = and Ocm{Q-a) = - ^) = 

£2 K-'Zqi (see Ifi.ll for this notation), by Appendix II this gives the exact sequence of cohomology 

S'H%C, 9^ -L)^ A'H\C, 

i=l i=l 
9-3 g-3 

H\X, Q.a) H\C, K-Y,q^-L)®H\C,K-Y,<l^-L)■ 

i=l 1=1 

Since h^Y^Qi) = 1. by Appendix O the elements of H%C^'^\e_a) = A^ H"^ [C , K - YiZl Qi) all 
vanish on fl C*-^-', hence they also vanish on fl X . So if the coboundary map is 

not injective, then there must be elements of H^{X, 0_a) which are not restrictions of elements of 
H^{C^'^\ B_a). In particular, by the above exact sequence, we must have H^{C, K-YJlZl Qi-L) ^ 
0. □ 

For r] E S'^H^Oc) \ H\Tc), define Q{ri), Z{ri), Z{ri) as in Paragraph O We have 

Theorem 5.2. Suppose X deforms infinitesimally with A in a direction t] G S'^H^{Oc) \ H^{Tc), 
then either d = 4 or d = 5 and h^{L) = 3. Furthermore, in this case we can choose Z{r]) to be of 
dimension at least g — A unless g > 7, C is a double cover of a smooth curve of genus 2 and L is 
the inverse image of the g\ on the curve of genus 2. 
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Proof. Let ^ G Z{ri) be general so that in particular we have hPi^Qi) = 1 fsee 14. ip . Then, as 
we noted in 14 .21 the image rj of 77 in H'^{Ozg_-^r]Xa{^)) is not zero. Since X deforms with 77, by 
Theorem 13. 2t the image of 77 in H^{Izg_^{Q)\xa) is zero. So is in the kernel of the coboundary 
map 

which is therefore not injective. It follows, by Lemma EHl that h^{K — ^ gj — L) > 0. 

Since the dimension of Z{ri) is at least (7 — 4 (see Proposition 14. 4p and X is one-dimensional, the 
dimension of Z{ri) is at least g — 5. If the genus is 4, then since Z{7]) is not empty, the dimension 
of Z{ri) is at least g — 4. 

If Z{ri) has dimension > (? — 4, then by the above discussion we have h'^lK — ^qi — L) > 
for a, {g — 4)-dimensional family of Yli=i 1i (i^ ^iv))- So h'^i^K — L) > g — 3 and, by Clifford's 
Lemma, since C is not hyperelliptic, we have 2((yf — 3 — l)<25f — 2 — (ior(i<5. If d = 5, then 
clearly /i°(L) = 3. 

Suppose now that every component Z{ri) has dimension g — 5. Then (? > 5 by the above and 
Z{ri) C Z{Xm) ■ Here Clifford's Lemma only gives us < 7 so we use the following argument. Since 
h^i^Qi) = 1 generically on Zlt]), the | Yl^lil form a. {g — 5)-dimensional family of linear systems 
and so do the \K — Y^qi\. Writing | -ft' — X] I — L + B', the 5' vary in a family of effective divisors 
of dimension > g — 5. Therefore the degree of B' is at least g — 5 and d+g — 5 < 2g — 2 — {g — 3), i.e., 
d < 6. Next Z{r]) has dimension g — 4 and the general fibers of Z^t]) — * Z^t]) are one-dimensional, 
all equal to a union of components of Xm, say X'. Since we can suppose h^{J2li) — ^ f^see 14. 
the condition h'^{D2 + E > 2 for all D2 G X' means that {J^qi) n {D2) ^ for all D2 G X'. 
Therefore the projection of center gj) from the canonical curve C to = \K — qi\* is not 
birational to its image. So there is a nonconstant map k, : C ^ C of degree > 2 with C smooth 
such that 

X' C {D2 G C^^) : 3t G C" such that D2 < K*t}. 

and 

\K-Y,<li\='^*N + Bo 
where is a two-dimensional linear system on C and Bq is the base divisor of \K — Y^Qil- 
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Consider now the linear systems \K — '^qi\. As ^ g'j varies in ^(77), the divisors of these hnear 
systems form a {g — 3)-dimensional family of divisors. Therefore we have 

g - 3 < deg{Bo) + deg{N) or deg{Bo) > g - 3 - deg{N). 

Combining this with the equality 

deg{Bo) + deg(K)deg{N) ^g + 1 

we obtain 

deg{N){deg{n) - 1) < 4. 
Since N has degree at least 2 we first obtain 

deg{K,) < 3. 

If deg{K) = 3, then deg{N) = 2 and C is a conic in P^. Hence C is trigonal and X' = X2{g\) = 
{D2 : h^{gl - D2) > 0}. In this case Z{X) = C^f^^^ since for any ^ C^f"'^', if we take 

D2 = gl — Qi ^ X' 1 then h^{D2 — ^^Qi) > 2. So Z{rj) = Z{X) is of dimension g — 3 which is 
contrary to our hypothesis. 

Therefore k, has degree 2 and X' — {K,*t : t e C'}. In this case, since C is not hyperelliptic, C is 
birational to a plane curve of degree 3 or 4 and has genus 1, 2 or 3. If C is elliptic, then any divisor 
Yli=i + '^*^ ^{v) ^iid Z{rii) is of dimension g — A which is against our hypothesis. If C has 
genus > 2, then its plane model has degree 4. If C has genus 3, then it has only one gl which is 
then A^. This implies that \K — k*N\ has dimension > g — 5 since {K — k* N — ^2 qi) > for Y^Qi 
in a (gf — 5)-dimensional family of effective divisors. Therefore h^{K*N) > 5 by the Riemann-Roch 
Theorem. However, this is impossible as \K — J^Qi] — Bq + k,*N is a complete linear system of 
dimension 2 for a general & Z{r]). 

So C has genus 2 and its plane model has a double point: N — g\-\-ti-\-t2 for some points ti 
and t2 on C such that ^1+^2^ qI,- We obtain deg^Bo) = g — 7 and, for G Z{r]) general, Bq 
is a general effective divisor of degree g — 7 on C. In particular, g > 7. Furthermore, since Bq is 
general and hP{K*N + Bq — L) > for all Bq, we obtain 



h^{K*N - L) > 0. 
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Now, since the \K — 'Y^qi\ vary in a family of dimension g — 5,N must vary in a family of dimension 
2, i.e., the points ti and ^2 are general in C. Since L is fixed this gives 

h\K*gl - L) > 

and L has degree 4. □ 

5.1. If (i = 4, then X is a Prym-embedded curve ^Hl- So X deforms out of J7g into the locus of 
Prym varieties. 

Let us then analyze the case d = 5. Here h^{L) = 3 so (7 < 6. By the above, for X to deform out 
of J'g, it is necessary that, if generically on a component of Z{X) we have h^{K — ^ gj — L) =0, 
then the image in Q of the inverse image of that component in Z{X) does not generate |/2(C)|. 
Let D2 & X he such that h^iY^Qi + -^2) ^ 2. Since L is in a g"^, any divisor of L spans a plane in 
Let us now distinguish the cases of different genera. 

g=4: Here g — 3 = 1 and X] ~ The variety Q = |/2(C)| is a point so for X to deform 
out of J'4 we need that for all qi G ^(X), /^^(fC — L — qi) > 0. Any (7^ on C is of the form 
\K — 1\ for some point t on C. So h^{K — L — gi) = — qi) is positive only when t = qi. 
So for X to deform out of J'g we need Z{X) = {t}. 

Let us now determine Z{X). To say h^{D2 + gi) > 2 means of course \D2 + qi\ is one of 
the two possibly equal g\ on C. Denote this g\ by G. So 1^2 is also on 

X2{G) = {D2:h\G-D2)>^}. 

First note that for any g\ on C, X2(5'|) = C is irreducible and if X contains it, then 
Z{X) = C and X cannot deform. Next the intersection number of X2(G') with X is 

X ■ X2(G') = (Cl - (5) ■ (Cg - 5) = 15 - 5 - 3 - 3 = 4. 

We now find these four divisors of degree 2 geometrically. The divisors of g^ = \K—t\ D L 
are cut by planes through t. A pencil of these planes whose base locus we denote by 
Lq C \K\*, [Lq = P^) cuts the divisors of L on C and the divisor of L containing D2 is 
cut by the span {Lq, t + D2) which is a plane. On the other hand, the divisors of G are cut 
on C by a ruling R of the unique quadric Q containing G. Since X does not contain X2{gl) 
for any g^ on G, it follows that Lq is not contained in Q. So LqCi Q is the union of two 
possibly equal points. Exactly one line of R passes through each of these points cutting 
two divisors of G on G. One of these divisors is the divisor of R containing t, say E2 + t 
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with E2 e Xr\X2{G). Writing the other divisor as ^1 + ^2 + ^3, we have ti + tj e Xr\X2{G) 
for all i, j e {1, 2, 3} which give us the other three points oi X f] X2{G). This means that 
ti G Z{X) for all i G {1, 2, 3}. Therefore for X to deform, we must have ^1 = ^2 = ^3= t. 
Therefore the two divisors of R are equal to 3t, in particular, Lq is tangent to Q. 

If C has another we repeat the above argument to obtain that it is also equal to |3t|. 

So we see that if X deforms out of J4, then C has only one gl with a triple ramification 
point t such that 5t G L C \K — t\ and X2{gl) meets X only at 2t with intersection 
multiphcity 4. Finally note that the facts gl — \3t\, 5t E L and X does not contain X2{gl) 
imply that L has no base-points. 
g=5: Here g — 3 = 2 and Qi = qi + q2- The linear system \K — L\ = \K — gf \ is a g^ on 
C, unique because the genus is > 5. The variety Q is a plane quintic with a double point: 
it is the image of C in = |-^2(C')| by the morphism associated to gf. Every quadric of 
Q has rank 4 except its double point Qo which has rank 3. The singular locus of Qo is a 
secant to C and its ruling cuts the divisors of gl on C. The intersection of the singular 
locus of Qo with C is the divisor Dq such that 2gl + Dq K. The base locus of IhiC)] in 
li^'l* is the rational normal scroll traced by the lines generated by the divisors of g^. 

To determine Z{X) we first fix a general divisor D2 G C*-^-* and find all the divisors 
?i + ?2 such that h'^{D2 + qi + q2) > 2. To say hP{D2 + qi + q2) > 2 means |-D2 + ?i + ?2| is a 
gl on C. To this gl is associated a quadric of rank 4 which then contains {D2) . Assuming 
^°(fl'3 ~ -^2) = 0, there is exactly a pencil of quadrics of \I2{C)\ which contain {D2). This 
pencil cuts Q in five points counted with multiplicities giving us five quadrics counted with 
multiplicities, and for each quadric a choice of a ruling containing {02). To each ruling is 
associated a gl such that h'^lgl — D2) > 0. These gl can be described as follows. Assuming 
that D2 ^ Dq, there is a unique divisor of gl which contains D2. Let this divisor be and 
write — D2 + si + S2 + S3. We have three gl containing D2 obtained as \D2 + Si + Sj\. 
Futhermore, if D2 — ti + 12, we have two other 514 containing D2 obtained as Ififs + It 
is not difficult to see that these are distinct for a general choice of D2. 

Since 0? > 4, we can find D2 E X such that hP{g\ — D2) =0. Taking such D2 general in 
X we can also assume D2 ^ Dq. With the above notation, the possibly equal elements of 
Z[X) that we obtain for D2 are Si + Sj and g\—ti. The last two are contained in a divisor 
of g\ — \K — L\, meaning they satisfy h^{K — L — Y^qi) > 0. The pair {D2, Si + sj) G Z{X) 
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is above Si + Sj G Z{X) and its image in Q is the quadric swept by the planes spanned 
by the divisors of \D2 + Si + Sj\. This quadric is also the image of Sk + gl for k ^ i,j 
since Sk + = \K — D2 — Si — Sj\. So the quadric is the image of the point Sk of C 
in Q. Since the base divisor of L has degree at most 2, for a general choice of D2 as 
above, at least one of the Sj will be a general point of C, and as D2 varies, this point will 
trace all of C and its image in Q will trace all of Q. So for X to deform we also need 
h^{K — L — Si — Sj) = h^{g\ — Si — sj) > for all i 7^ j. This implies si + S2 + S3 G gl 
and since the divisor Si + S2 + S3 is not fixed, we obtain L = g^ + D2. This contradicts the 
generality of D2. Therefore X cannot deform out of jTs. 
g=6: Here g — 3 = 3 and J^^li = I1 ~^ + Qs- The curve C is a smooth plane quintic and 
K ~ 2g1 ~ 2L. The variety Sing{Q) is the image of C x C via (p, q) ^ Ig^ — p + q\ (see 
e.g. [2j p. 264). So every complete gl on C has exactly one base point. Since C embeds 
in by the map associated to g'^, given ti + t2 = D2 E X , there is a unique divisor 
-D5 = -D2 + Si + S2 + S3 of gl containing it. The one-parameter family Z{D2) of gl such 
that h'^igl — D2) > has six components: one component is the family of pencils in g'^ 
passing through D5, two components are families of complete gl obtained as \gl — t\ + ti 
where t varies in C, and the last three components are families of complete gl obtained as 
I-D2 + Sj + Sj + 1\ with t varying in C. So altogether (and counting multiplicities) Z{D2) is 
the union of a smooth rational curve and 5 copies of C. The divisors ^ qi for the rational 
component are all equal to Si + S2 + S3 for which h^{K — L — qi) = h^igi — Si — S2 — S3) = 
hP{D2) > 0. The divisors X] ^"^^ ^^^^ t^*^ copies of C in Z{D2) are gl — t — tj so 
we see that they satisfy h^{K - L-^qi) = h^{gl - {gl - t - tj)) = h^{t + tj) > 0. The 
divisors ^ q^ for the last three copies of C in Z{D2) are Sj + Sj + 1 and so for t general, 
we have h^{K — L — J^qi) = ^^idi ~ Si — Sj — t) = h^{D2 + Sk — t) = 0. As we saw, here 
'^qi = Si + Sj + t = gl — D2 — Sk+t. The divisors that we obtain in Z{X) map in Sing{Q) 
to gl — Sk + 1. As D2 varies in X, the points s^ and t vary freely in C and gl — s^ + t traces 
all of Sing{Q). So we see that X cannot deform with JC out of J^^. Note however, that 
if we degenerate the plane quintic to a singular one, then X might deform. 



6. Appendix 
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6.1. The cohomology of some sheaves on C^"). We calculate the cohomologies of some sheaves 
on C*^") for an integer n > 2. Recall that 7r„ : C" — > (7^"^ is the natural morphism and let A"^ 
(1 < i < j < n) be the diagonals of C". Also let pr^ : C" ^ C be the i-th projection. Then 

l<i<j<n 

by the Hurwitz formula, and 

ujcn = (8)"=ipr*a;c. 

For any non-trivial divisor class b of degree on C, the intersection 0.0^ is easily seen to be 
reduced and its inverse image in IS 

{D e C^^-^) : h%D -b)>0} = {De C^'-^^ : h\K + 6 - > 0}. 

If n < 51 — 1, the restriction of this to C^]_i_„ is 

g-l-n 

{D e C(") : h\K + b- J2 qi-D)>0} 

i=l 

whose pull-back to C" by 7r„ is in the linear system 

\prlOc{K + b- J2 ^i)^--- 

i=l 

g—l—n 

...®prlOc{K + b- q,)®Ocr^{- 

1=1 l<i<j<n 

as can be easily seen by restricting to fibers of the various projections — > C^~^ and using the 
See-saw Theorem. 

More generally, for any divisor E on C, let Cn,E and C^^^ be the invertible sheaves on C^'^^ 
whose inverse images by tt^ are isomorphic to 

and 

prlOciE) . . . pr:Oc{E) Oc4- E 

l<i<j<n 

respectively. We will calculate the cohomologies of Cn,E and g. 
Since 

n:Cn,E = prlOciE) ® . . . ® pr^ME), 
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the sheaf Cn^E is the invariant subsheaf of 



T^n*KCn,E = TlnMlOc{E) ® . . . ®pTlOc{E)) 



for the action of the symmetric group ©„. We claim that ^ is the skew-symmetric subsheaf of 



for the action of 6„. To see this, first note that any skew-symmetric local section of 'Kn*{WiOc{E)® 
■ ■ ■®Wn'^c{E)) vanishes on the diagonal A" of C^'^\ Conversely, puUing back to C", we see that 
the ideal sheaf of any of the diagonals is generated by skew-symmetric tensors. 

Therefore the cohomology groups of Cn,E (resp. £'j ^) are the invariant (resp. skew-symmetric) 
parts of the cohomology groups of T^n^ipf^iOciE) ® ... ® pr^Oc{E)) under the action of (3„. 
Or, equivalently, since 7r„ is finite, the invariant (resp. skew-symmetric) parts of the cohomology 
groups of prlOc{E) ... pr^Oc{E) under the action of 6„. By the Kiinneth formula the 
cohomology groups of prlOc{E) (8) . . . ® pr^Oc{E) are 



where the n-uples (ji, . . . , j„) describe the set of n-uples of elements of {0, 1}, i of which are equal to 
1 and the rest equal to 0. The action of 6„ on each of these cohomology groups is super-symmetric: 
for instance any transposition r exchanging I and k sends an element ei®. . .(8)e;(8). . .06^(8). . .<8)e„ 
to (— l)-''^'=ei ® . . .®ek® . . .®ei® . . .®en. Prom this one easily calculates that the invariant parts 
of the cohomology groups are 



TlnM*lOc{E) ® . . .®pr*nOc{E)) 



H\prlOc{E) ® . . .®prlOc{E)) = J] W'{E) ® . . . ® W-{E) 



H\£^^e) = H\E) (g) S^-^H\E) 
H''{Cn,E) = A''H\E) S^-''H\E) 



H^-\Cn,E) = A"- 

H''{Cn,E) = A«i/^ 



-^H\E)(^H^>{E) 
\E). 
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Similarly, the skew-symmetric parts of the cohomology groups are 

H^iC'^ j,) = S^H\E) ® A^-^H\E) 

^""HCb) = S''-^H\E) ® H^{E) 
H-{a^^j,)^S-H\E). 

6.2. Useful exact sequences of cohomology groups. Let a be such that 6a D C^^^ and 
Sing{Qa) ~^ C*^^^ Then —a + XI ~ X] Q'j ^^"^ = 1- we saw in 16.11 we have 

0(^(2) (6a) = i^'2K~Y,qi' Consider the composition 

For 3 < n < g — 1, we have the exact sequence 

O^O^W (6a-C^;_\| )-^OMn) (6a) 

^C>„(„-1, (6a)— ^0. 

For each bv 16. ![ we have 

H\0 {n) (6a - C'f^s-i^, )) = 

^ g-l-n l^i = l 

^ S'H\K - 5^ g.) ® A^-'H\K -J^^^)^ 

i=l i=l 

g—l—n g—l—n 

H\0^(,., {Qa)) = S'H\K- V g,)®A"-*/7°(ir- V g,), 

i=l i=l 
g-n g-n 

H\0^„.^., (6a)) = S'H\K -Y^Qi)® A^-'-'H\K - V g,) 

^i = l ''l 1=1 1=1 



and the map on cohomology 



i/^(0^(„) {Qa-d":;-! ))-^H\0^,„, (6a)) 

g_i_„ qi g_l_„ 



is obtained from the inclusion 



g—n g—l—n 
i=l i=l 
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(note that the dimension of H^{K — Yli=i 1i) ^^"^ H^{K — Yli=i~^ 1i) follows that for all 

i the sequence 

is exact. In particular, all the sections of O^m (©a) vanish on {Zg^i)ar\C^'^\ hence on (Zg_i)anX, 
so they restrict to sections of X(2g_i)anx(0a) on X. 

6.3. The cohomology of C2,e{-^) = Oc(2){Ce - A) = Oc(2){Ce - A). We use the exact 
sequence 

Ocm{CE - A) ^ Ocm{CE) ^ Ocm{CE)U . 

Under the isomorphism A = C we have 0(j(2){Ce)\/i — Oci^E)) and, bv 16.11 we have the long 
exact sequence of cohomology 

— > H%C^^\ Ce - 25) — > S^H^{C, E) — > H^{C, 2E) — > 

(6.1) — > H\C^^\ Ce - 25) — ^ if°(C, E) ® H\C, E) — > 

— > H\C, 2E) — > H\C'^^\ Ce - 25) — > A^H\C, E) — ^ . 

So, in particular, H%C^^\Ce - 25) = hiC^E) is the space of quadratic forms vanishing on the 
image of C in \E\* if \E\ ^ 0. 

Note that using our result in Appendix 16.11 above, Pareschi and Popa [Hj have computed the 
cohomology of £„_£;(— A) for n > 2 as well. 
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